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and received self-paced instructional treatments. Learners were assessed on 
performance (achievement) , instructional efficiency, and efficiency of 
learner strategies employed (process efficiency) . Achievement measures for 
all groups showed significantly higher levels of achievement when compared to 
the appended control group. Using multivariate and univariate analyses of 
variance, data were examined to determine what effect varying instructional 
strategies had on learning. Results indicated no statistically significant 
differences between any treatment groups. However, some strong data trends 
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This study examined the effects of varying instructional strategies on learning in emerging technology- 
based visual literacy instruction. The present study used a completely crossed 2 (organizational strategy) x 2 
(delivery strategy) x 2 (management strategy) factorial design , post-test only control group design. Organizational 
strategies employed included criterion- referenced, objectivist instruction and problem-based, constructivist 
instruction. Delivery strategies employed included local, computer-based instruction and networked hypermedia- 
based (WWW) instruction. Management strategies implemented involved participation by individuals as well as 
cooperative dyads. 

One hundred thirty-eight undergraduate education students were randomly assigned to eight treatment 
groups and a control group and received self-paced instructional treatments. Learners were assessed on 
performance (achievement), instructional efficiency, and efficiency of learner strategies employed (process 
efficiency). 

Achievement measures for all treatment groups showed significantly higher levels of achievement when 
compared to the appended control group. Using multivariate and univariate analyses of variance, data were 
examined to determine what effect varying instructional strategies had on learning. Results indicated no statistically 
significant differences between any treatment groups. However, some strong data trends were observed. Data 
trends indicate further research should prove worthwhile in determining effects of instructional strategies on 
emerging technology-based instruction. 



Introduction 

Background and Theoretical Perspectives 

Educators today debate the most appropriate instructional role of technology, particularly computer 
technology. The past ten years have seen the development of two trends with unprecedented effects on the course of 
educational technology: an increase in the number and types of technology resources available, and dramatic shifts 
in beliefs about the fundamental goals and strategies of education itself (Robyler, Edwards, & Havriluk, 1997). As 
the goals of education begin to change to reflect new social and educational needs, teaching strategies also change, 
and so do strategies for integrating technology into teaching and learning. One of the newer technologies available 
for instruction is the emerging information system called the Internet and the graphical-based access called the 
World Wide Web. 

The effectiveness of newer interactive technology usage in instruction is open to debate. Moore, Myers, 
and Burton (1993) state that little actual research regarding the effectiveness of interactive technologies has been 
conducted and what has been conducted has limited value. Practitioners and researchers call for research that goes 
beyond traditional comparison studies. Grabowski and Pearson (1988), Slee (1989), Reeves (1986), Kozma (1991), 
and Moore et al( 1993) have made calls for conducting research on the instructional strategies related to the specific 
attributes of the interactive medium. Hannafin (1992) identified the areas of constructivist models of teaching and 
acquiring knowledge, hypermedia, and cooperative 

teaching and learning as innovative instructional strategies that need to be addressed in emerging technology 
research. 
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Instructional Strategies 

Gagnd, Briggs, and Wager (1992) define instructional strategy as “...a plan for assisting the learners with 
their study efforts for each performance objective (p. 27)." Dick and Carey (1996) define instructional strategy as an 
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overall plan of activities to achieve an instructional goal. Kemp, Morrison, and Ross (1994) differentiate 
instructional strategy from delivery strategy. Instructional strategy, they offer, prescribes sequences and methods of 
instruction to achieve an objective, and delivery strategy describes the general learning environment. Seels and 
Richey (1994) define instructional strategies as, “...specifications for selecting and sequencing events and activities 
within a lesson” (p. 31). 

In contrast, Reigeluth (1983), Merrill (1983), and Reigeluth and Merrill (1978) identify three components or 
variables of instructional strategies: organizational strategies, delivery strategies, and management strategies. 
Although this instructional strategy lexicon is not widely used, it is this all-encompassing view of instructional 
strategies that is used as a framework for this study, as indicated in Table 1, below. 

Organizational Strategies 

Organizational strategies are those decisions the instructional designer makes when designing learning 
activities. Most important of these decisions is how the designer will assist learners to process new information and 

to process at a deeper level, 
producing meaningful learning. 
Most often, this is 
accomplished by the 
presentation and sequencing of 
content. How content will be 
presented and sequenced is 
most often determined in 
response to what type of 
learning is to take place, and 
how the designer believes an 
individual learns. Further 
examination of organizational 
strategies requires one to examine, at the most philosophical level, their belief whether or not knowledge exists 
independent of the learner. Which epistemological approach to learning a designer espouses will have great impact 
on the organizational strategy selected for use. At one end of the epistemological continuum is objectivism, while at 
the other is constructivism (Jonassen, 1991a; Jonassen, Wilson, Wang, & Grabinger, 1993). 

Thede (1995) compared constructivist and objectivist frameworks in the design of computer-assisted 
instruction dealing with basic chemistry instruction for nursing students. The objectivist form of instruction was 
loosely criterion referenced, while the constructivist form was problem-based, but in a closed environment. Thede 
found subjects receiving the objectivist treatment scored significantly higher than the constructivist/ problem-based 
group on objectivistic post-test, particularly on the recall level questions. The objectivist group also scored higher on 
comprehension and application level questions, but not significantly so. The treatments varied widely in their 
presentation of what Thede termed identical content. In addition, no constructivist evaluation was given. 

Savery and Duffy (1995) consider problem-based learning (PBL) to be one of the best exemplars of a 
constructivist learning environment. Very little research has yet been conducted on the effectiveness of problem- 
based learning as an instructional strategy (Gallagher, Stepian, & Rosenthal, 1992). Most of the research on PBL to 
date has focused on the adjustment of faculty to an innovative technique, (McAuley & Woodward, 1984; Moore- 
West & O’Donnell, 1985; Neame, 1982), student perception of the technique’s usefulness (Blumberg & Daugherty, 
1989), and the mechanisms of programmatic change (Thompson & Williams, 1985). Little, if any research has been 
conducted to date on the ways in which problem-based instruction affects student learning and cognition. 

Many studies comparing discovery learning and expository learning have been reported. Because this 
comparison approximates somewhat the differences between constructivism and objectivism, these studies bear 
review. The results of these discovery-expository comparison studies are inconclusive. Andrews (1984), Hall and 
McCurdy (1990), Haukoos and Penick (1987), Lovelace and McKnight (1980), and Sugrue and Thomas (March, 
1989) found in favor of discovery learning. In other studies, no significant differences were found by Canino and 
Cicchelli (1988), Elshout and Veenman (1992), Gesi, Massaro, and Cohen (1992), and Haukoos and Penick (1983). 

Delivery Strategies 

In the design of instruction, delivery strategies are those decisions affecting the way the information will be 
carried to the learner. Delivery strategies affect the selection of the instructional media that will be used to present 
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the- learning activities (Merrill, 1983). This study focuses on two delivery strategies: local computer-based 
instruction, and as indicated by Hannafin (1992), networked hypermedia based instruction on the World Wide Web. 

Computer-Based Instruction 

Several theories have been the basis for investigating the effect of computers in the teaching and learning 
process, including behaviorism, systems theory, and cognitive theory. Most of the techniques applied to the design 
and use of CBI can be traced to one of these theories (Simonson & Thompson, 1997). 

In an attempt to design effective instruction, early media research compared one medium with another, 
typically comparing a new medium with traditional instruction. This type of research focused on finding the ’’best” 
medium of instruction for teaching students regardless of the learning situation. Overall, this research represented an 
effort to determine which medium would achieve the greatest with the greatest number of learners. 

Although this research was prevalent, some researchers indicated that this type of investigation was 
inappropriate as a major research direction. For example, Allen (1959) stated that enough comparative effectiveness 
research had already been conducted, and that in part, research should focus on discovering the unique educational 
characteristics of each of the media of instruction. He reiterated this view in a later article (Allen, 1971) adding that 
such research should also include the relationship of media characteristics or attributes to the characteristics of the 
learner and the nature of the instructional task. A decade later, Clark (1983) also made a convincing case for 
research that emphasized instructional methods incorporated into media rather than studies aimed at finding the best 
medium for instruction. Instead of focusing on media, Clark (1975), Levie and Dickie (1973) and Saloman (1974, 
1979) suggested that we study “attributes” of media and their influence on the way that information is processed in 
learning. Clark (1983) explained that it wasn't a particular media that offered advantages to the learning situation, 
but more a particular attribute of that medium that was not peculiar to only that medium, but shared by other media. 

For the purpose of this study, this researcher has identified the following basic attribute categories for 
computer-based and networked hypermedia-based instruction (NHBI): access, content presentation, interactivity, 
and locus of control. Defining design attributes of CBI and NHBI are displayed in Table 2, below. 

Table 2. Defining Attributes of CBI and NHBI. 
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Networked Hypermedia-Based Instruction 

An emerging type of technology-based instruction is networked hypermedia-based instruction (NHBI), 
commonly referred to as Web-based instruction. Hypermedia-based instruction is the application of hypermedia 
techniques to educational software (Heller, 1990). When hypermedia-assisted instruction or hypermedia-based 
instruction is delivered over a network, or more specifically, the Internet, it is called networked hypermedia-based 
instruction. Networked hypermedia-based instruction (NHBI) is referred to by a variety of terms and abbreviations, 
including WBT (Web-based training), WBI (Web-based instruction), and LBT (Internet-based training) (Driscoll, 
1997). 

NHBI is similar to traditional multimedia computer programs. In NHBI, learners engage in self-paced 
learning programs that use multimedia to communicate content. Interactions take place in the form of branching 
decisions that are controlled by either the learner or the program. These programs are most frequently used to 
transfer knowledge, build comprehension, and practice applying skills (Driscoll, 1997). The simplest form of NHBI 
programs are electronic books that feature hypertext and images. More sophisticated NHBI programs can include 
audio, video, and animation. 
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NHBI differs from CBI in its capacity to use the Internet and provide communication links (Driscoll, 1997; 
Heller t 1990). NHBI’s greatest advantage, therefore, is access to the multitude of resources available on the World 
Wide Web. The node and link structure allows that, in NHBI, instruction is not self-contained into one application 
package. Because of this, more than one document may be open at one time as the content may be spread across 
many network sites. 

This node and link structure is one defining attribute of NHBI. A hypertext learning environment such as 
NHBI involves semantic nodes which are linked by an author for potential learner use. The semantic nodes are often 
the kind of word that might be indexed in an ordinary document. Instead of being listed at the rear of the document, 
such words are linked to other instances or other associated words in the same document or in other documents. In 
NHBI, these links can take the learner to another document located on another server anywhere in the world. By 
pointing and clicking on such words, the learner can travel across links to related sources of information. Other 
characteristics that distinguish NHBI from its precursor CBI are given in Table 1, above. 

Little research has been done to date in the area of hypermedia in general. Jonassen (1991b) states, “to 
date, there is virtually no research that supports or rejects semantically structured hypertexts” (p. 93). Frey and 
Simonson (1990) report a study on cognitive style, perceptual modes, anxiety, attitude, and achievement in a 
hypermedia environment. However, hypermedia was simply used as a treatment vehicle and its relationship to other 
variables was not examined. Reeves and Harmon (1989) examined the relationship between a person’s creativity 
and hypermedia, but found no significant relationship between the two variables. Jonassen (1991c) reports a study 
exploring graphical browser node arrangement in the form of an expert’s semantic map. A control group provided 
with no structural information regarding nodes reported higher recall scores than treatment groups which were 
provided with descriptions of the nature of the links between nodes. The results of the study seem to indicate that 
hypermedia may not be as effective when highly structured. 

Management Strategy/Task Structure 

Management strategies are those decisions made in the design of instruction that affect the way the 
individual student will be helped to interact with the learning activities (Merrill, 1983). They are decisions about 
which organizational and delivery strategy components to use during the instructional process (Reigeluth, 1983). 
Management strategies involve motivational techniques and individualization schemes, among other implementation 
activities (Merrill, 1983). How learners will receive instruction, whether by lecture, individual seatwork, in small 
groups, and so on, is an example of management strategy. This study examines two management strategies: 
individual reception of instruction, and instruction delivered to cooperative groupings of two, or dyads. 

Individualized Instruction 

Instruction designed to meet the needs of the individual learner has long been a driving force in 
instructional design. Instructional design reminds us that regardless of how instruction might be delivered, be it to 
one learner, to twenty learners or to 400 learners, learning still occurs within individuals. Bloom (1984) maintains 
that one-to-one tutorial instruction is the most effective form and that an average student in a tutorial program 
achieves more than 98 percent of students in conventional classroom instruction. The history of instructional 
technology also indicates a strong commitment to individualized instruction which accommodates self-pacing, 
prescribes instructional methods according to learner characteristics, and permits content selection according to 
individual goals (Reiser, 1987). The rationale for individualization in technology-based instruction is intuitively 
appealing: people differ in ability, background, readiness, motivation, interest, and learning style and may therefore 
require different types and different paces of instruction to enhance their learning (Hooper, 1992a). Much evidence 
supports the belief that optimum learning takes place when the learner works at his or her own pace, is actively 
involved in the learning activities, and experiences success in learning (Kemp, Morrison, & Ross, 1994). 

Cooperative Learning 

In contrast, to increase the occurrence of active participation on the part of the learner, Slavin (1983) 
recommends structuring the learning environment to include other types of learner interactions, particularly student- 
student interaction, so as to provide more of an opportunity for students to take an active part in their learning. 
Cooperative task structures refer to those situations when group of two or more individuals can or must work 
together but may or may not receive rewards based on their group's performance. Virtually all cooperative groups 
incorporate a cooperative task structure (Hooper, 1992a) where group members work together toward a common 
task. 
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Research on cooperation has been conducted since the beginning of this century (see, Mailer, 1929), 
although classroom research on practical cooperative methods began in the early 1970s. Despite the many studies 
done, the effects of cooperation on performance are still rather poorly understood (Slavin, 1983). Johnson and 
Johnson (1974), in a meta-analysis, found that cooperation is better than individualization or competition for all but 
the most concrete and repetitive tasks. Johnson, Maruyama, Johnson, Nelson, and Skon (1981) found strong 
evidence supporting cooperative incentive structures over individualistic or competitive incentive structures. In 
contrast, Michaels (1977) reviewed much of the same literature and concluded that competition is usually better than 
cooperation for most tasks. 

The research seems more conclusive when examining cooperation and problem solving. As early as the 
1930s, Thorndike (1938) considered the superiority of group to individual problem solving to have been proved. 
Studies that examined the reasons that groups did better on problem-solving tasks concluded that they did better 
simply because of the pooled problem-solving abilities of their members (Slavin, 1983). Faust (1959), Marquart 
(1955), and Ryack (1965) compared groups that really worked together to groups that groups that were randomly 
assigned to artificial groups. In all three studies, the real groups had higher achievement than the individuals, but not 
than the artificial groups, suggesting that the real groups had high scores not because of interaction, or motivation, 
but because if any individual in the group could solve the problem, than all members in the group would get credit 
for it, regardless of their participation or learning (Slavin, 1983). 

Sufficient interaction is also a problem in computer-based instruction. Some believe that individualized 
instruction presented by computer creates “sterile’ 1 learning environments where learners interact only with the 
computer, apart from teachers and peers (Hannafin, Dalton, & Hooper, 1987). Dalton (1990) comments that, for 
many, “individualized” is synonymous with “isolated.” 

Research indicates that students who complete CBI in cooperative groups generally perform as well as 
students who work alone (Carrier & Sales, 1987; Shlechter, 1990) and often perform better (Dalton, Hooper, & 
Hannafin, 1989; Hooper, 1992b; Johnson, Johnson, & Stanne, 1985, 1986). Research supports cooperative learning 
as a means to improve achievement and attitudes in computer-based learning environments. Dalton and Hannafin 
(1987) found that learners working in pairs significantly outperformed learners working individually during a CAI 
lesson. In addition, learners working cooperatively on a CAI lesson produced a greater quality and quantity of daily 
work and demonstrated greater problem-solving skills than learners working individually or competitively (Johnson, 
Johnson, & Stanne, 1985). 

Often, the effectiveness of cooperative learning is attributed to interaction among group members (Webb, 
1989). Therefore, efforts should be made to promote interaction in cooperative learning situations. Hooper and 
Hannafin (1991) state one method to promote interaction involves increasing individual accountability, wherein 
each group member must demonstrate mastery of content in the instruction. This technique might also be used to 
isolate and remediate potential learning problems within the group and to provide an additional incentive to 
cooperate (Hooper, Ward, Hannafin, & Clark, 1989), as well as motivate more able group members to provide help 
and less able group members to invest sufficient mental effort to master instruction (Kerr, 1983; Kerr & Bruun, 
1983). 

Problem Statement 

Hannafin’s (1992) call for research for examining the effects of varying instructional strategies on 
technology-based instruction served as the basis for this study. The purpose of this study was to examine the effects 
of varying instructional strategies on learning in technology-based visual literacy instruction. Specifically, this 
study examined the effects of instructional organizational strategies, instructional delivery strategies, and 
instructional management strategies on learner performance in technology-based visual literacy instruction. 
Interaction effects of these instructional strategies on learner performance were also examined. Learning was 
assessed in three forms: achievement, instructional efficiency, and process efficiency. For the purpose of this study, 
instructional efficiency was operationally defined as the level of achievement (combined achievement score) divided 
by the amount of time spent on treatment. Process efficiency was operationally defined as a measure of the 
efficiency of the process of learning in terms of learner note-taking strategies employed in learning. 



Methods 

Subjects 

Subjects in the present study consisted of 138 volunteers from a pre-service media methods course at a 
large midwestern university. The course is a required course for all teacher education students in the College of 
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Education. All subjects were undergraduate students, primarily sophomores and juniors. They ranged in age from 19 
to 46 with a mean age of 22.1 years. The gender makeup of the sample population was 34% male and 66% female. 
In addition, subjects represented a variety of intended major areas of study. 

Materials 

Materials consisted of four separate visual literacy instructional packages representing two levels of 
instructional organizational strategies and two levels of instructional delivery strategies: a criterion referenced local 
CBI lesson, a problem-based local CBI lesson, a criterion referenced NHBI lesson, and a problem-based NHBI 
lesson. Treatment materials were developed by a design team consisting of one CBI designer, one NHBI designer, 
and two content advisers/subject matter experts, one of which was the investigator. The investigator had three years 
teaching experience with this particular subject matter in the course which served as the subject pool. The second 
content adviser/subject matter expert was a faculty member in the college and the course supervisor with 15 years 
experience in the field. 

Objectivist Criterion Referenced Treatments 

The criterion-referenced modules loosely followed Gagne’s (1968, 1985) recommended events of 
instruction. Prior to the commencement of the lesson, subjects were given a one page introduction to the lesson that 
informed them that there would later be a quiz on the content in the lesson. An introductory section attempted to 
capture the learner's attention by mentioning and showing familiar visual images. At the opening of each section of 
content, subjects were informed of the learning objectives for that section on screen. The information, or stimulus 
material was next presented, followed the learning hierarchy. To allow learners some control, some sequencing of 
content was permitted, but the menu selections on both the CBI and NHBI versions were arranged in a suggested 
hierarchy as portrayed by a vertical screen menu. When faced with a vertical menu, subjects tend to select menu 
options in a top to bottom order. On-screen questions allowed learners to practice the new content. General feedback 
on the correct response was provided on screen for learners. 

Constructivist , Problem-Based Treatments 

The problem-based treatments were identical in content to the criterion referenced treatments. The 
differences included visual portrayal of menu selections on the local CBI treatments, no advising the subject of 
performance objectives, and the direction the subject received prior to commencing the lesson. The menu screen of 
the local CBI problem-based treatment package was arranged in a chevron shape that implied no inherent order of 
the content. This was done to encourage the subject to explore topics in order of interest or need to solve the 
problem presented, and to avoid imposing a hierarchical order of content on the learner. Due to limited options for 
presenting linked lists, the NHBI problem-based version of the module used a bulleted menu list identical to that 
used in the criterion-referenced NHBI module. 

Learners were not informed of the instructional objectives prior to the presentation of content. This was 
done to allow learners to construct their own schemas for the content, rather than have an order imposed by the 
learning objectives. Instead, learners were presented with a teaching-related problem much like one that they would 
face in the real world. Learners were informed that they must create a poster and an overhead transparency to help 
them teach an instructional design topic that they had been developing. Learners were also informed that they would 
need to use the media developed in a teaching demonstration to their peer class. It was recommended that learners 
familiarize themselves with visual literacy concepts to aid them in their media development. No further explanation 
was given. 

Local Computer-Based Instructional Treatments 

The computer -based instructional program was developed using Authorware© on Macintosh personal 
computers. Every attempt was made to adhere to recommended screen design guidelines as given in Alessi and 
Trollip (1991) and Schwier and Misanchuk (1994). 

The program included an optional introductory section on how to navigate through the program with the 
use of iconic buttons. This section was made optional so as not to bore the more technologically competent user. 
However, the design team felt that including these directions might facilitate comfort in the novice user, thereby 
focusing their attention on the content, rather than the distraction of operating the program. 

The locus of control rested largely with the program in terms of content presentation. Learners were 
allowed to control the sequencing of content through menu selection. Once a learner was within a content section. 





however, they had only the option of proceeding to the next screen in the sequence, backing-up to the preceding 
screen^ or returning to the menu screen. 

In all but a few instances, the learner controlled the pace of screen presentation. However, there were 
instances when the program controlled the revelation of text on a screen in order to draw the learner’s attention to 
specific points within that section of content. Text was never removed from the screen without the learner pressing 
the forward button; instead text was "dimmed" on revelation screens to cue the learner to look at new, appearing 
information. 

The amount of content presented on each screen was kept to a limit of one idea or content “chunk” per 
screen. A total of 218 screens make up the criterion referenced CBI program, while 212 screens make up the 
problem-based package (less screens due to the lack of objective presentation). In order to keep the length of the 
program under one hour, opportunities for learner practice were limited. At the end of several sections, however, 
learners were asked thought provoking questions in text and visual formats. As visual literacy is a highly visual 
topic, an effort was made to include as many visual representations as possible. The final CBI package included 162 
scanned images, clip art images, and hand-drawn graphics. All images and artwork included in the program were 
copyright-free. All content, information, and practice questions were contained in the CBI program. No outside 
materials besides the introduction page were made available to the students. 



Networked Hypermedia-Based WWW Treatments 

The networked hypermedia based instructional program was developed using identical text and images 
from the CBI version. Text and images were copied and formatted into HTML documents. As screen design 
guidelines for Web-based documents are lacking, attempts were made to make the pages as user-friendly as 
possible. 

No introductory section on how to navigate through the program was included, as learners had previously 
studied navigation on the WWW. Learners were reminded on their introduction pages that links were presented in 
an alternate color to non-linked text, and that clicking on a link would take them to additional information on the 
topic. The NHBI version of the lesson consisted of 21 separate HTML page documents. The 21 pages included 53 
links: nine links connecting to another area of the same page; 38 links connecting to a different page within the 
lesson, and six links connecting to sites external to the lesson. 

The locus of control rested largely with the learner in terms of content presentation. Learners were allowed 
to control the sequencing of content through a list of links. The Web pages were planned to be used with a browser 

such as Netscape Navigator® or Microsoft Explorer®, that in addition to links, would allow the user to back up to 
all previously viewed screens in any order. 

The learner controlled the pace of screen presentation through the use of scrolling pages. Revelation was 
not an option in the design of the Web pages, and so was not included. Text was never removed from the screen 
without the learner scrolling. For most of the sections of content, all content for that section was contained in one 
scrolling page. Sub-menu level lists linked the user to separate pages for design elements and design principles. 

The final NHBI package included all 162 scanned images, clip art images, and hand-drawn graphics that 
were included in the local CBI package. However, many of the images were reduced to smaller sizes so as to reduce 
delays caused by downloading large image files from the Internet. 

The HTML documents were loaded onto a Macintosh server. The Web pages were accessible from any 
type of personal computer with Internet access (Macintosh or PC). The server was tested to ascertain the number of 
users or “hits” it could accommodate at one time. It was determined that the upper limit of access was twelve 
simultaneous users, which was deemed acceptable for the size of the treatment groups in this study. For the purposes 
of this study, the Web pages were placed on the server only during treatment runs (so as to control exposure to 
subjects assigned to NHBI groups with Internet access outside the treatment time frame). All content, information, 
and practice questions were contained in the NHBI program. In addition, five pointer links were included to sites 
relating to visual literacy elsewhere on the Internet. Users were familiar with performing searches on the Internet, so 
additional information could be sought at the learner’s discretion. 



Dependent Measures 

Learners were assessed on their achievement, instructional efficiency, and process efficiency. Achievement 
for this study was a combined score of a 30-item objective post-test and a 15-item checklist of the application of 
visual literacy concepts to learner generated visual media. 
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Achievement Objective Assessment 

The achievement posttest was designed and produced by the researcher. Based on the instructional content 
of the visual literacy lesson, the instrument measured the knowledge gained by participants during their use of the 
instructional materials. The same test was given to all treatment groups and to the appended control group. The 
instrument was administered to participants two weeks after exposure to the treatments in order to measure long- 
term retention of knowledge. 

The instrument covered all sections of the visual literacy content and was composed of 30 multiple choice 
questions with four possible answers to each question. The first 25 questions were entirely text-based, while the 
remaining five questions asked the learner to examine or interpret a given visual in order to generate a response. 
Items were written by the researcher based on the instructional objectives set forth in the instructional design. A 
typical item included on the instrument was: “Age, gender, race, and past experiences are examples of a) visual 
filters; b) socioeconomic filters; c) cultural filters; d) personal filters.” The instrument was intended to serve as an 
objectivistic measure of achievement. 

The content validity of the instrument was checked through the inspection of questions by subject matter 
experts in the field of visual literacy who were asked to check for overall content and accuracy. Using data obtained 
during the study, the instrument was examined for internal consistency. After examining the intercorrelations of 
item score and total score on the objective assessment instrument, six questions were dropped due to low correlates. 
Reliability of the remaining 22-item showed a Cronbach’s coefficient alpha of 0.65 (n items = 22). 

Achievement Application Assessment 

Jonassen (1992) recommends for constructivistic learning that more goal-free evaluation methods be used 
and that it is higher order thinking skills that should be assessed. In response to this, a second achievement 
assessment instrument was designed and produced by the researcher. This second instrument, was designed to 
measure the application of visual literacy concepts and design principles, rather than simple recall. This application 
assessment was based on the instructional content of the visual literacy lesson to measure the knowledge gained and 
applied by participants after their use of the instructional materials. The same assessment was performed on all 
treatment groups . Participants were given two weeks after treatment to produce an instructional poster and an 
overhead transparency. The instrument assessed participant production in order to measure application of visual 
literacy knowledge. 

The instrument covered all sections of the visual literacy content and was composed of 15 items or 
categories with five possible ratings in each category (none, poor, moderate, good, and excellent). A typical item for 
rating on the application assessment was: “Apparent eye path (use of COZULST layout)”. The instrument was 
intended to serve as a constructivist measure of achievement. 

As with the objective assessment of achievement instrument, the content validity of the application 
instrument was also checked through the inspection of categories by subject matter experts in the field of visual 
literacy who were asked to check the instrument for overall content and accuracy. Using data obtained during the 
study, the reliability of theTifteen item constructivist checklist which was applied to the subject-generated poster and 
overhead transparency yielded a Cronbach’s coefficient alpha of 0.95 (n items = 30). 

Combined Achievement Measure 

As a variable with low reliability may include errors of measurement which may attenuate F ratios and 
generally confound interpretations of experimental effects, Pruzek (1971) recommends combining highly similar 
measures. In keeping with this recommendation, the 22-item pencil and paper objective measure and the two 
applications (one for each the poster and the overhead transparency) of the 15-item checklist were treated as a 
combined achievement score. For the 113 subjects, the internal consistency of the combined achievement measure 
calculated a Cronbach alpha of 0.93 (n items = 52). 

Instructional Efficiency 

Instructional efficiency was defined as the combined achievement post-test scores divided by the time 
required to complete the lesson. Units for this measure were given as score per minute. 



Process Efficiency 

' Process efficiency was operationally defined as a measure of the efficiency of the learner strategies (note- 
taking) employed during the instruction. Process efficiency is calculated as the total achievement score multiplied by 
time spent on treatment and divided by the quantity of notes taken (word count). 



Procedure 

All subjects in this study were randomly assigned by computer to one of eight treatment groups as part of 
the required curriculum of the course: 



• OLI: 

• OLC: 

• OWI: 

• OLC: 

• PH: 

• PLC: 

• PWI: 

• PWC: 



Objectivist (Criterion referenced) local CBI, individual; 

Objectivist (Criterion referenced) local CBI, cooperative grouping; 
Objectivist (Criterion referenced) Web-based (NHBI), individual; 

Objectivist (Criterion referenced) Web-based (NHBI), cooperative grouping; 
Constructivist (Problem-based) local CBI, individual; 

Constructivist (Problem-based) local CBI, cooperative grouping; 
Constructivist (Problem-based) Web-based (NHBI), individual; and 
Constructivist (Problem-based) Web-based (NHBI), cooperative grouping. 



Subjects experienced treatment during their individual class meetings. At the treatment session, subjects were 
directed to their assigned computer workstation, asked to complete a demographic questionnaire, and to read the 
introduction handout. Both the CBI package and the NHBI Web site were pre-loaded for the subject’s convenience. 
Subjects assigned to cooperative groupings were directed to wait for their assigned partner to commence the 
treatment. 

All subjects were supplied with a notebook in which to record notes for study purposes. Notebooks were 
encouraged as generative learning aids such as note taking and outlining require the learner to take an active and 
responsible role in the instruction/learning process (Jonassen, 1988; Wittrock, 1974). The notebook was intended to 
promote deeper processing of the content, and to keep subjects from simply scrolling through the lesson. The subject 
was instructed to record their start time and ending time on the cover of their notebook. Notebooks were collected 
after treatment, photocopied, and returned to the subject in the next class period. 

Content review via computer required between 30 and 100 minutes and was, for the most part, completed 
during the regularly schedule 50 minute class period. Students voluntarily stayed after class time in order to 
complete the modules. After completion of the instructional treatment, subjects were dismissed. The subjects then 
had two weeks in which to produce their instructional media. Subject notebooks were returned so that they could use 
their notes for reference and study purposes. The treatment materials (CBI and NHBI packages) were not available 
for reference. 

Two weeks after treatment, section instructors collected subject-created instructional media. At that time, 
each subject completed the 30-item objective achievement instrument during a regular class meeting. Completion of 
the in-class assessment instrument required approximately fifteen minutes. 

Subject instructional media productions were digitally imaged for later assessment by the researcher and 
returned to the subjects for further course use. The researcher reviewed the digitally imaged media productions, 
using the application achievement instrument to assign, each subject and application achievement score. Scoring of 
all measures was performed blind to treatment group assignment 

In addition to the 138 subjects, and additional 15 subjects volunteered to serve as an appended control for 
the study. The 15 control subjects were students in the same course during a later semester. The appended control 
group, without the benefit of instruction, completed the objective measure and produced both a poster and an 
overhead transparency. The application measure checklist was again used to quantify the application of visual 
literacy concepts to the production of educational media. The appended control group served as test of the reliability 
of the instructional materials. 



Experimental Design and Data Analysis 

The present study primarily used a completely crossed 2x2x2 (2^) factorial design, post-test only control 
group design with organizational strategy, delivery strategy, and management strategy as the independent variables. 
The first independent variable, organizational strategy, had two levels — a constructivist, problem-based approach to 
instruction and an objective oriented, criterion referenced approach to instruction. The second factor, delivery 



strategy, also had two levels — via local computer-based instruction and networked hypermedia-based instruction 
via the' World Wide Web. The last independent variable, management strategy, was also measured at two levels of 
individualization schemes, individual subjects and subjects in cooperative groupings of two. 

The dependent variables for this study included a combined score of two measures of achievement, a post- 
test, and an evaluation of the application of visual literacy principles to the design of two pieces of educational 
media (a poster and an overhead transparency). In addition, instructional efficiency as measured by total 
achievement divided by time spent on treatment was a second dependent variable. Process efficiency, a measure of 
the efficiency of the learner strategies (note-taking) employed during the instruction, was a final dependent variable. 
Process efficiency is calculated as the total achievement score multiplied by the quantity of notes taken (word count) 
and divided by the time spent on treatment. The factorial matrix used is portrayed visually in Figure 1. 
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Figure J. 2x2x2 Factorial Design of Independent Variables 

The relationship between dependent variables was assessed by examining correlations between dependent variables 
and by examining scatter plots of dependent variable scores to assess the direction of any relationships. Preliminary 
data analysis for this study was accomplished through MANOVA procedures. MANOVA effects were followed 
with univariate analysis of variance procedures to determine effects of and interactive effects of the organizational 
strategies, delivery strategies, and management strategies on each of the three dependent variables. 

Results 

Relationship Between Dependent Variables 

Intercorrelations were run between the dependent variables to determine the relationship between 
achievement, instructional efficiency, and process efficiency. The Pearson-Product Moment correlation coefficient 
of combined achievement scores and instructional efficiency, r = 0.815, shows a strong, direct relationship between 
the two variables. The Pearson Product Moment correlation coefficients between combined achievement and 
process efficiency, and between instructional efficiency and process efficiency, r = 0.599 and r = 0.63, respectively, 
show a moderate, direct relationship. 

Scatter plots of the dependent variables were generated to visually inspect the relationship between 
dependent variables to ascertain any nonlinear or curvilinear relationships. The relationship between 
achievement and instructional efficiency was strong, linear, and positive. This is to be expected as 
instructional efficiency is the dividend of achievement divided by time spent on treatment. The relationship 
between achievement and process efficiency was moderate, positive, and somewhat linear. The relationship 
between instructional efficiency and process efficiency as moderate, positive, and again, somewhat linear. 

Results Regarding Research Questions 

Multivariate and univariate analyses of variance were used to analyze data. The multivariate analysis of 
variance addresses all main effects and interaction effects of the instructional strategies examined in this study on 
the dependent variables as a whole. In contrast, the univariate analyses of variance address the main effects and 
interaction effects of the instructional strategies examined in this study on each dependent measure individually. 

Multivariate Analysis of Variance 

As the three dependent variables are correlated and likely share a common conceptual meaning, a three- 
way multivariate analysis of variance was used to analyze the data. The principle advantage of such a multivariate 
procedure over the traditional univariate F tests is that it permits a test of the possible interactions among multiple 



dependent measures that cannot be evaluated if each dependent measure is r tested in isolation (Huck, Cormier, & 
Bound.s, 1974). 

The Wilk’s Lambda criterion for multivariate analysis showed there is no significant multivariate effect 
attributable to organizational strategy (F (3, 95) = 0.765, ps <0.52), delivery strategy (F (3, 95) = 0.2.271 ps <0.09), 
or management strategy ( F (3, 95) = 0.940, ps <0.43). The multivariate effect for the interaction of organizational 
strategy and delivery strategy, F (3, 95) = 2.348, is not significant at ps < 0.08. In addition, the multivariate 
difference for the interaction between organizational strategy and management strategy, and delivery strategy and 
management strategy can be accounted for by chance alone (Fs = 1.32 and 0.43, ps < .27 and .73, respectively). 
Finally, the multivariate effect attributable to the interaction of all three factors, organizational strategy, delivery 
strategy, and management strategy, F (3, 95) = 1.619 is not significant at ps < 0.19. A summary of the multivariate F 
ratios appears in Table 3. 



Table 3. Multivariate Analysis of Variance (MAN OVA) of Achievement, Instructional Efficiency, and Process 
Efficiency by Organizational Strategy, Delivery Strategy, and Management Strategy 
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Univariate Analysis of Variance 

Although the multivariate analysis of variance for all three dependent measures showed no significant main 
or interaction effects generated by the instructional strategies studied, the strong but not significant effect of 
delivery strategy on the dependent measures (F (3,95) = 2.27, ps <.09) as well as the similarly strong but not 
significant interaction effects between organizational strategy and delivery strategy (F (3, 95) = 2.35, ps <0.08) and 
the moderate interaction effect between organizational strategy, delivery strategy, and management strategy (F (3. 
95) = 1.619, ps<0.20) merits additional examination. Means and standard deviations for dependent measures for all 
treatment groups are shown in Table 4. 



Achievement 

The univariate main effect of organizational strategy on achievement, F = 1.77, df- 1/97, although strong, was not 
significant at p < 0.19. Varying instructional organization strategies (criterion referenced instruction and problem based 
learning) showed no significant effect on achievement when learning from technology based visual literacy instruction. 
Although not significant, examination of the cell means in Table 4 shows that for all groups, those groups receiving the 
constructivist problem-based treatment consistently scored higher on combined achievement than did those groups receiving 
the objectivist, criterion- referenced treatment. The relationship between organizational strategy and achievement is shown in 

The univariate main effect of delivery strategy on achievement, F = 0.07, df = 1/97, is not significant at p < 0.79. 
Varying instructional delivery strategies (local computer-based instruction (CBI) and networked hypermedia based (WWW) 
instruction (NHBI)) shows no significant effect on achievement when learning from technology based visual literacy 
instruction. Although not significant, examination of the cell means in Table 4 shows that for all groups, those groups 
receiving the networked hypermedia based instruction consistently scored higher on combined achievement than did those 
groups receiving the local computer-based treatment. 

The univariate main effect of management strategy on achievement, F - 0.001, df = 1/97, is not significant at p < 
0.97. Varying instructional management strategies (individual and cooperative task structures) shows no significant effect on 
achievement when learning from technology based visual literacy instruction. Although the effect of management strategy on 
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instructional efficiency is not significant, examination of the cell means in Table 4 shows that for all groups, the individual 
task structure groups consistently showed a higher achievement score than did the cooperative task structure groups. 

Table 4. Cell Means and Standard Deviations Comparing Organizational Strategy , Delivery Strategy , and 
Management Strategy for Combined Achievement, Instructional Efficiency, and Process Efficiency 
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Dependent Variable: Instructional Efficiency fecore/tnin.) 
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Dependent Variable: Process Efficiency 
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Figure 2. Combined Achievement Means by Organizational 
Strategy and Delivery Strategy 



The univariate interaction effect of organizational strategy and delivery strategy, organizational strategy 
and management strategy, and delivery strategy and management strategy, show no significant effect on 
achievement. In addition, the interaction effect of organizational strategy, delivery strategy, and management 
strategy is also not significant (F = 1.6, df= 1/97, p < 0.21). The instructional strategies studied do not significantly 
interact during instruction to effect achievement when learning from technology based visual literacy instruction. 

Instructional Efficiency 

No significant main or interaction effects of organizational strategy, delivery strategy, or management 
strategy on instructional efficiency were indicated by the statistical analysis. Although the effects of organizational 
strategy, delivery strategy, and management strategy on instructional efficiency are not significant (F = .1.67, df = 
1/97, p < 0.2; F = 2.36, df - 1/97, p < 0.13, and F = .617 df - 1/97 , p < 0.43, respectively), examination of the cell 
means in Table 4 shows trends similar to those observed with the dependent measure, achievement. For all groups, 
those groups receiving the problem-based treatment consistently showed a higher level of instructional efficiency 
than did those groups receiving the objectivist treatment. In a similar vein, those groups receiving the networked 
hypermedia-based treatment consistently showed a higher level of instructional efficiency than did those groups 
receiving the local computer-based treatment. Finally, the individual task structure groups consistently showed a 
higher level of instructional efficiency than did the cooperative task structure groups. 

Process Efficiency Results 

No significant main or interaction effects of organizational strategy, delivery strategy, or management 
strategy on process efficiency were indicated by the statistical analysis. Although the effect of organizational 
strategy on process efficiency is not significant (F = 1.77, df - 1/97, p < 0.19), examination of the cell means in 
Table 4 shows that for all groups, those groups receiving the objectivist CBI and objectivist NHBI treatments 
performed similarly. In comparison, the mean process efficiency of those groups receiving the problem-based CBI 
decreased slightly from their objectivist counterparts. The mean process efficiency of the problem-based NHBI 
group increased strongly, indicating a probable drop in the number of notes taken with no corresponding drop in 
achievement score or time on treatment. The relationship between organizational strategy and instructional 
efficiency is shown in Figure 3. 
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Figure 3. Process Efficiency Means by Organizational 
Strategy and Delivery Strategy 



The effect of delivery strategy on process efficiency is not significant (F = 3.22, df = 1/97, p < 0.08). As 
this is the strongest F ratio of the entire study, this relationship bears some further examination. Examination of the 
cell means in Table 15 shows that for all groups, all groups receiving the local computer-based treatment and the 
groups receiving the objectivist networked hypermedia-based treatment consistently showed similar levels of 
process efficiency. Those groups receiving the problem-based networked hypermedia-based treatments showed 
considerably higher levels of process efficiency. 

The effect of management strategy on process efficiency is also not significant (F = .1.37 df = 1/97, p < 
0..24). However, examination of the cell means in Table 4 shows that for all groups, the individual task structure 
groups consistently showed a higher level of process efficiency than did the cooperative task structure groups. 

Discussion 

Discussion of Research Findings by Organizational Strategy 

MANOVA and ANOVA results indicated that there were no statistically significant differences in the 
amount of achievement, instructional efficiency, or processing efficiency associated with instructional organization 
strategies (criterion referenced instruction and problem based learning) when learning from technology based visual 
literacy instruction. Interesting trends, however, were observed in the data associated with organizational strategy. 
Groups receiving the constructivist problem-based treatment consistently scored higher on combined achievement 
that did those groups receiving the objectivist, criterion-referenced treatment. The corresponding F statistic 
associated with the univariate main effect of organizational strategy on achievement, F = 1.77, df = 1/97, although 
not significant, is strong at p < 0.19. 

Those groups receiving the problem-based treatment also consistently showed a higher level of 
instructional efficiency that did those groups receiving the objectivist treatment. This was not expected, as the 
problem-based treatment gave learners no recommended path to review content. However, they showed no increase 
in time which would have decreased their instructional efficiency. 

A similar type of relationship is seen between organizational strategy and process efficiency. Groups 
receiving the objectivist CBI and objectivist NHBI treatments performed similarly. In comparison, the mean process 
efficiency of those groups receiving the problem-based CBI decreased slightly from their objectivist counterparts. 



The mean process efficiency of the problem-based NHBI group increased strongly, indicating a probable drop in the 
number of notes taken with no corresponding drop in achievement score or time on treatment. 

The design of treatments employing different organizational strategies was tightly controlled to maintain 
equivalent content and content organization. In other words, although some effort was made to make the objectivist, 
criterion-referenced treatment and the constructivist, problem-based treatment dissimilar, a strong effort was also 
made to ensure that any effect seen would not be due to a difference in the amount or nature of content received, a 
problem noted by Mielke (1968) in early media-based research. If these restrictions were loosened somewhat, 
thereby increasingly polarizing the treatments, more significant differences in the achievement, instructional 
efficiency, and process efficiency levels of the treatment groups might be observed. 

Discussion of Research Findings by Delivery Strategy 

MANOVA and ANOVA results indicated that there were no statistically significant differences in the 
amount of achievement, instructional efficiency, or processing efficiency associated with varying instructional 
delivery strategies (local computer-based instruction and networked hypermedia-based instruction) when learning 
from technology-based visual literacy instruction. It should be noted that of all the instructional strategies examined, 
delivery strategies showed the strongest effect on the dependent measures. The multivariate analysis of variance 
showed the main effect generated by delivery strategies on all dependent variables combined was strong, but not 
significant at F (3, 95) = 2.27, ps <0.09. 

The univariate main effect generated by delivery strategy on achievement was not strong at F =0.07, df = 
1/97, p < 0.79. Groups receiving the networked hypermedia-based treatment consistently showed a higher combined 
achievement score and showed a higher level of instructional efficiency than did those groups receiving the local 
computer-based treatment. This was not expected as the researcher assumed the networked hypermedia-based 
treatment would require greater time to access and download individual pages. In retrospect, the learners did not 
have to wait to move from one screen of chunked content to the next as they did with the local computer-based 
treatment, so this likely accounted for any decrease in time spent on the treatment, thus increasing instructional 
efficiency. 

A similar type of relationship is seen between delivery strategy and process efficiency. Groups receiving 
the local computer-based treatment and the groups receiving the objectivist networked hypermedia-based treatment 
showed similar levels of process efficiency. Those groups receiving the problem-based networked hypermedia- 
based treatments showed considerably higher levels of process efficiency, indicating a possible interaction effect 
between organizational strategy and delivery strategy on process efficiency. 

Although there are an increasing number of WWW-page design guides (see, for example, Apple Web 
Design Guide, 1996; Schwier & Misanchuk, 1996, February), little has been written giving recommendations or 
guidelines for creating instructionally efficient WWW pages. Indeed, much of educational use of the Internet has 
been archival in nature (lecture notes, slide shows, and visual representations) (Shotsberger, 1996). With no 
recommended instructional screen design guidelines, the design team sought to create instructionally effective Web 
pages, extrapolating several screen design guidelines from computer based instruction and balancing them with 
measures to decrease download time of pages and images within pages. Hannafin (1985) proposes that the more 
rapidly program segments are accessed, the more effective the instruction. In order to keep the level and nature of 
content presentation similar across treatments, it is likely that the design team did not take full advantage of the node 
and link structure of the Internet. This minimal levels of links available most likely also decreased the level of 
disorientation felt by subjects, as is often the case in Web-based learning (Tripp & Roby, 1990). In addition, visual 
images were presented in a smaller format than they were on the local CBI treatments in order to decrease download 
time. As was the case with organizational strategies, increased polarization of the delivery strategy treatments might 
result in more significant differences in the achievement, instructional efficiency, and process efficiency levels of 
the treatment groups. 
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Discussion of Research Findings by Management Strategy 

.7 MANOVA and ANOVA results indicated that there were no statistically significant differences in the 
amount of achievement, instructional efficiency, or processing efficiency associated with varying instructional 
management strategies (individual and cooperative task structures) when learning from technology-based visual 
literacy instruction. Groups receiving instruction in an individual task structure consistently showed only a slightly 
higher combined achievement score than did those groups receiving instruction in cooperative dyads. This supports 
earlier research by Carrier and Sales (1987) and Shlechter (1990) that found students who complete CBI in 
cooperative groupings generally perform as well as students who work alone. 

In addition, those groups receiving instruction in individual task structures consistently showed a higher 
level of instructional efficiency that did those groups receiving instruction in cooperative dyads This is to be 
expected, as an interaction, meaningful or otherwise, would cause the cooperative groupings to spend more time on 
the instructional treatment, thereby decreasing instructional efficiency. Instructional efficiency was considerably 
lower for those groups receiving the objectivist instructional treatment in a cooperative task structure. 

A similar type of relationship is seen between management strategy and process efficiency. Groups 
receiving instruction in individual task structures once again showed a higher level of process efficiency than those 
groups receiving instruction in cooperative dyads. Since the instructional efficiency was higher for groups receiving 
instruction individually, indicating they spent less time on the treatment, a higher level of process efficiency for the 
individual groups indicates that the individual groups took less notes than the cooperative dyads, resulting in a 
higher level of process efficiency. 

Discussion of Research Findings by Interactions of Instructional Strategies 

Of all the interaction combinations of instructional strategies possible, only organizational strategy and 
delivery strategy showed any potential of an interaction effect on any dependent variable. 

The multivariate analysis of variance showed the interaction effect generated by organizational strategy and 
delivery strategy on all dependent variables combined was nearly significant at F (3, 95) = 2.34, ps <0.08. 
Univariate analysis of variance showed the interaction effect generated by organizational strategy and delivery 
strategy on achievement to be not significant (F = 0.15, df- 1, 97. p <0.7). Univariate analysis of variance showed 
the interaction effect generated by organizational strategy and delivery strategy on instructional efficiency also to be 
not significant ( F = 0.07, df - 1, 97. p <0.79). 

In contrast, univariate analysis of variance showed the interaction effect generated by organizational 
strategy and delivery strategy on process efficiency to be nearly significant (F = 3.26 df - 1 , 97. p <0.7). Examining 
Figure 3, a visual display of process efficiency means by delivery strategy and organizational strategy, the 
interaction direction can be seen. 

The relationship can be described as follows. Groups which received the objectivist treatment showed 
relatively the same level of process efficiency regardless of delivery strategy. Groups which received the objectivist, 
networked hypermedia-based treatment also demonstrated approximately the same level of process efficiency. The 
groups which received the problem-based, networked hypermedia based treatments, however, displayed 
substantially higher levels of process efficiency. Since instructional efficiency does not show a similar potential 
interaction effect, it can be surmised that the potential interaction effect is likely due to a decrease in the quantity of 
notes taken by these groups, with no accompanying decrease in achievement scores or change in the amount of time 
spent on treatment. 

The reasoning for this potential interaction effect can only be hypothesized. One reason may be that as 
proposed by Borsook and Higginbotham-Wheat (1992, February), the node and link structure of networked 
hypermedia-based instruction may in some way mirror the linked structure of mental schema. Kearsley (1988) offers 
that hypertext or hypermedia systems like the World Wide Web, improve learning by focusing attention on the 
relationships between ideas rather than on isolated facts. The associations provided by the links in NHBI, Kearsley 
offers, should facilitate retrieval, concept formation, and comprehension. If this is the case, further exploration of the 
application of constructivist learning theory to the design of Web-based instruction is certainly called for. 



Implications and Recommendations 

These results indicate that instructional strategies may be an important variable in learning from 
technology-based instruction. However, further research is needed to determine which aspects of each type of 
instructional strategy effects learning in emerging technology-based instruction. In particular, the treatments used in 
this study need to be refined in order to increase the differences between factor levels, thus increasing the power of 
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the test. In addition, additional variables need to be researched, including aptitude, self-directedness, and motivation. 

Although this study offered no statistically significant results, it does open the door for many research 
paths. At the very least, the lack of effect between treatments indicate that alternative methods and technologies of 
teaching and learning, including constructivist learning, networked hypermedia-based instruction, and cooperative 
learning deserve additional study. 
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